Maximizing the Margin with Feed forward Neural Networks

Feed forward Neural Networks FNNs and Support Vector Machines SVMs are two ma chine learning frameworks developed from very dif ferent starting points of view In this work a new learning model for FNNs is proposed such that in the linearly separable case tends to obtain the same solution that SVMs The key idea of the model is a weighting of the sum of squares error function which is inspired in the AdaBoost algorithm The model depends on a parameter that controls the hardness of the margin as in SVMs so that it can be used for the non linearly separable case as well In addition it allows to deal with multiclass and multilabel prob lems in a natural way as FNNs usually do and it is not restricted to the use of kernel functions Finally it is independent of the concrete algorithm used to minimize the error function Both theoretic and ex perimental results are shown to con rm these ideas